fix(tools): isolate RAG auto-detection across items and support uppercase extensions by truecallerabreham · Pull Request #7828 · crewAIInc/crewAI

Abreham Melese (truecallerabreham) · 2026-09-30T07:11:33Z

Related issue

Summary

In CrewAIRagAdapter.add(), when data_type was not explicitly provided, the loop reassigned the outer data_type parameter to the first item's detected type (data_type = DataTypes.from_content(...)). Subsequent iterations saw if not data_type: as False and forced that type onto every subsequent item. Calling rag_tool.add(directory, file) crashed with ValueError: Directory does not exist: <file> because the file was validated as a directory.

In addition, DataTypes.from_content() checked extensions with path.endswith(ext) without lowercasing, causing uppercase extensions (.PDF, .JSON, .CSV, .DOCX) to fall back to DataType.TEXT_FILE. In TextFileLoader, decoding binary .PDF or .DOCX files with UTF-8 crashed immediately with UnicodeDecodeError. For URLs, uppercase extensions fell back to DataType.WEBSITE.

This fix:

Replaces the outer variable mutation in CrewAIRagAdapter.add() with a loop-local item_data_type, so each item in content_items is auto-detected independently.
Lowers path before checking suffix mappings in DataTypes.from_content().
Adds regression tests covering uppercase file extensions, uppercase URL extensions, and multi-item auto-detection without type leakage.

Verification

Tests added or updated for the changed behavior
Relevant tests and quality checks pass locally

Added TestAutoDetectMultipleItemsAndCaseSensitivity to lib/crewai-tools/tests/tools/rag/test_rag_tool_add_data_type.py:

test_auto_detect_uppercase_file_extension
test_auto_detect_uppercase_url_extension
test_auto_detect_multiple_items_independent_types

All 54 tests in lib/crewai-tools/tests/tools/rag/ pass locally.

Additional context

None

Note

Low Risk
Scoped bugfix in RAG ingestion auto-detection with new regression tests; no auth, security, or persistence schema changes.

Overview
Fixes two bugs in RAG add() auto-detection that broke mixed batch ingestion and uppercase filenames.

Per-item type detection: When data_type is omitted, the adapter no longer writes the first item’s detected type back into the shared data_type variable inside the loop. Each content item now uses a loop-local item_data_type, so calls like add(directory, file) no longer treat the file as a directory (which raised ValueError: Directory does not exist).

Case-insensitive extensions: DataTypes.from_content() lowercases the path before suffix matching, so extensions such as .JSON, .PDF, and .CSV resolve to the correct loaders instead of falling back to text/website handling (which could trigger decode errors on binary formats).

Regression coverage adds tests for uppercase paths/URLs and directory-plus-file batches without type leakage.

^{Reviewed by Cursor Bugbot for commit 7b1c87a. Bugbot is set up for automated code reviews on this repo. Configure here.}

…case extensions In CrewAIRagAdapter.add(), when data_type was not specified, the loop reassigned the outer data_type variable on the first item, permanently locking its type onto all subsequent items. This caused tool.add(directory, file) to crash with ValueError: Directory does not exist on the file. Additionally, DataTypes.from_content() checked suffixes using path.endswith() without lowercasing, causing uppercase extensions like .PDF and .JSON to fall back to DataType.TEXT_FILE (which crashes binary files with UnicodeDecodeError in TextFileLoader) or DataType.WEBSITE for URLs. Fixes crewAIInc#7827

coderabbitai · 2026-09-30T07:12:01Z

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 31b627ec-0ac5-43e0-90b6-178e02ad610f

📥 Commits

Reviewing files that changed from the base of the PR and between a0d16dd and 7b1c87a.

📒 Files selected for processing (3)

lib/crewai-tools/src/crewai_tools/adapters/crewai_rag_adapter.py
lib/crewai-tools/src/crewai_tools/rag/data_types.py
lib/crewai-tools/tests/tools/rag/test_rag_tool_add_data_type.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📝 Walkthrough

Walkthrough

RAG ingestion now detects mapped file extensions without case sensitivity. It also resolves the data type separately for each content item and uses that type when processing and recording the item.

Changes

RAG content type detection

Layer / File(s)	Summary
Case-insensitive extension detection `lib/crewai-tools/src/crewai_tools/rag/data_types.py`, `lib/crewai-tools/tests/tools/rag/test_rag_tool_add_data_type.py`	File type detection lowercases paths before matching mapped extensions. Tests cover uppercase `.JSON` files and uppercase `.PDF` and `.CSV` URL extensions.
Per-item data type resolution `lib/crewai-tools/src/crewai_tools/adapters/crewai_rag_adapter.py`, `lib/crewai-tools/tests/tools/rag/test_rag_tool_add_data_type.py`	Each content item uses its own data type for directory handling, file validation, loader and chunker selection, and stored metadata. A test covers adding a directory followed by a file.

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 7b1c8

The mixed-item regression test would fail with the previous behavior, and no actionable merge risk is established.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 3 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes both main fixes: per-item RAG auto-detection and uppercase extension support.
Description check	✅ Passed	The description includes the related issue, problem analysis, solution summary, verification details, test coverage, and additional context. It satisfies the repository template.
Linked Issues check	✅ Passed	The changes satisfy issue `#7827`. `CrewAIRagAdapter.add` resolves `item_data_type` for each item when no explicit type is provided, so later items do not inherit the first item type. Explicit non-`FILE…
Out of Scope Changes check	✅ Passed	The changes remain within issue `#7827`. The source changes update per-item RAG type detection and case-insensitive extension matching. The added tests directly verify these behaviors. No unrelated impl…

Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Abreham Melese (truecallerabreham) · 2026-09-30T07:24:28Z

cc Vidit Ostwal (@Vidit-Ostwal) João Moura (@joaomdmoura) whenever you have a moment to review.

This addresses #7827 by ensuring RAG auto-detection uses an item-local data type instead of mutating the outer loop variable across multi-item inputs, and makes extension matching in DataTypes.from_content case-insensitive. Regression unit tests are included and all checks are passing.

github-actions · 2026-11-14T12:19:01Z

This PR is stale because it has been open for 45 days with no activity.

Abreham Melese (truecallerabreham) mentioned this pull request Sep 30, 2026

[BUG] RAG auto-detection leaks data_type across multiple items and misses uppercase file extensions #7827

Closed

Lorenze Jay (lorenzejay) mentioned this pull request Oct 13, 2026

[FEATURE] Wire JSONL into RAG DataType discovery and ingestion #8131

Closed

5 tasks

ellaslan11 mentioned this pull request Oct 20, 2026

feat(rag): route Jupyter notebooks through RAG ingestion #8307

Open

Aadesh Salecha (AadeshSalecha) mentioned this pull request Oct 29, 2026

feat(tools): wire jsonl into rag data type discovery #8754

Merged

2 tasks done

github-actions Bot added the no-pr-activity label Nov 14, 2026

Navigation Menu

Repository navigation

fix(tools): isolate RAG auto-detection across items and support uppercase extensions - #7828

fix(tools): isolate RAG auto-detection across items and support uppercase extensions#7828
Abreham Melese (truecallerabreham) wants to merge 1 commit into
crewAIInc:mainfrom
truecallerabreham:fix-rag-data-type-auto-detection

Abreham Melese (truecallerabreham) commented Sep 30, 2026 •

edited by cursor Bot

Loading

Uh oh!

coderabbitai Bot commented Sep 30, 2026 •

edited

Loading

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

Abreham Melese (truecallerabreham) commented Sep 30, 2026

Uh oh!

github-actions Bot commented Nov 14, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Abreham Melese (truecallerabreham) commented Sep 30, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related issue

Summary

Verification

Additional context

Uh oh!

coderabbitai Bot commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

Abreham Melese (truecallerabreham) commented Sep 30, 2026

Uh oh!

github-actions Bot commented Nov 14, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Abreham Melese (truecallerabreham) commented Sep 30, 2026 •

edited by cursor Bot

Loading

coderabbitai Bot commented Sep 30, 2026 •

edited

Loading